This document analyses the self-developed scheduler component in Android-based mobile devices as part of the SyMptOMS project.

Reproduce notebook

To create the PDF of the computational notebook you can run the following commands in a new R session. If you have problems rendering the PDF you can execute each chunk independently in RStudio.

This document does not install the required R packages by default. You can run the script install.R to install all required dependencies on a new R installation, or use install.packages(..) to install missing R packages.

The plots and tables use the packages ggplot2, knitr::kable() and kableExtra.

Required libraries and runtime environment description are as follows.

library(tidyverse)
library(kableExtra)
library(here)
# library(googledrive)
library(lubridate)
library(stringr)
library(scales)
# library(plotly)
library(grid)
library(gridBase)
library(gridExtra)
library(egg)

Data download

Log files are stored in shared gdrvie folder. This notebook downloads them locally in the folderdata-raw.

Table. List of raw data files.
experiment gsheets_name gsheets_link
#1 AD_BQ.csv https://drive.google.com/open?id=1fP9r0S8ORa689yHRtfCtYQothLuGcCw3
#1 AD_NV.csv https://drive.google.com/open?id=1l-vnaT-Smy0SegArIz5A108-IQ2_jmVV
#1 AD_A1.csv https://drive.google.com/open?id=1gzk1ezN5t5yBv2RnbznUqvalDx97eu3o
#1 BA_H9.csv https://drive.google.com/open?id=1BLXO4Rvz6ppWFJe5oN9Y0hKh__ypofxk
#1 BA_MO.csv https://drive.google.com/open?id=1RizGjKR8QLOdJna7qGPP7dYX9WWAysO4
#2 BA_BQ.csv https://drive.google.com/open?id=1rfaooLZ0Up0gNx1SCA92TV1JFxcLU0sk
#2 BA_NV.csv https://drive.google.com/open?id=1_KIXX5FGXv7MO37QbUDnd_KGqiGihiMI
#2 BA_A1.csv https://drive.google.com/open?id=1LezrtyB9i4K7cQsmI78E0v_KaOyhZcXI
#2 AD_H9.csv https://drive.google.com/open?id=1ISjsfSLZt6n9miryQtDr-tsZV9bfbCDT
#2 AD_MO.csv https://drive.google.com/open?id=1bx2RVNz05qVKGr-32GI-ZIOKQWlqKBPz

Data preparation

Raw data files are cleaned and merged together into a single file stored in the folder data. New fields such as dates breakdown are computed too.

Key Variables:

Exploratory analysis

Summary statistics

Table. Descriptive statistics of data distribution and centrality per scheduler/device.
# exp Type scheduler Device # records Start End Min Bottom whisker 1st Qu. Median Mean 3rd Qu. Top whisker Max
#1 Advanced BQ Aquaris V 21391 2020-03-30 12:15:03 2020-04-14 13:07:57 -48.591 0.235 0.546 0.651 0.769 0.754 1.065 61.204
Nvidia Shield Tablet 21453 2020-03-30 12:15:03 2020-04-14 13:08:14 -52.811 -0.882 0.052 0.241 0.673 1.043 2.518 103.849
Xiaomi Mi A1 21465 2020-03-30 12:23:27 2020-04-14 13:07:50 -60.925 -0.840 0.088 0.269 0.493 0.838 1.945 68.872
Basic Honor 9 21574 2020-03-30 12:00:03 2020-04-14 13:01:35 -0.824 -0.094 0.068 0.087 0.311 0.187 0.364 295.963
Motorola Moto G 21619 2020-03-30 12:00:07 2020-04-14 13:01:16 -0.183 -0.183 0.062 0.100 0.875 0.289 0.629 60.896
#2 Advanced Honor 9 19377 2020-04-15 11:00:02 2020-04-29 12:59:15 -59.535 -0.423 0.167 0.225 0.591 0.842 1.853 93.032
Motorola Moto G 19735 2020-04-16 12:09:53 2020-04-30 12:58:44 -30.166 -0.212 0.117 0.167 2.092 0.782 1.778 126.487
Basic BQ Aquaris V 20146 2020-04-15 11:16:26 2020-04-29 13:00:20 -0.111 0.065 0.280 0.297 0.390 0.424 0.639 34.412
Nvidia Shield Tablet 3528 2020-04-15 11:16:18 2020-04-17 22:27:06 -0.871 -0.871 0.021 0.122 0.431 1.007 2.085 4.841
Xiaomi Mi A1 10832 2020-04-15 11:16:23 2020-04-23 00:52:00 -0.905 -0.661 0.066 0.302 0.405 0.614 1.416 108.035

Observations:

  • 181.120 records in total.

  • 50.6% (91.582/181.120) is the interquartile range or IQR, i.e. the middle 50% of the data.

  • 39.6% (71.769/181.120) represent the reasonable extremes of the data, i.e. these whisker values are the minimum and maximum values that do not exceed 1.5 × IQR.

  • 9.8% (17.769/181.120) are outliers, i.e. values beyond the top and bottom whiskers.

  • Nvidia Shield Tablet has stopped triggering alarms on 2020-04-17 22:27:06 when running the basic scheduler. Since there’s no watchdog in the basic version, this device is not going to trigger alarms anymore. Died!

  • Xiaomi Mi A1 has stopped triggering alarms on 2020-04-23 00:52:00 when running the basic scheduler. An explanation may be that Huawei/Honor smartphones on Android 9+ have a Launch Records where the OS records how much times an app has been launched by itself, so the OS has the possibility to detect this kind of apps. In these smartphones, Huawei also introduced a new task killer (Power Genius) which kills everything that is not whitelisted by Huawei, and does not give users any configuration options. Therefore, the Power Genius uses the information from the Launch Records to kill the scheduled alarms

Which one of the analysed schedulers is more reliable? (Distribution of outliers)

In mental health intervetions, to have a predictable execution of planned actions is preferred since these notifications and/or alerts are delivered at the right moment. It is inconceivable, for example, that a patient gets a notification (action) 1 minute later than the event (she walks past a gambling hall) that triggered it, because the patient could be dozens of meters away from the place of interest or, even worse, walked into that place. Therefore, closing the gap between the triggered event and the action is critical (time matters), especially because time and space/place must be in synchrony. If these tww diemsion are not in synchrony, the intervetion becomes less effective.

Given that average response time (delay) is similar in all devices (< 1second), outliers are a good candidate to study reliability. Fewer outliers would mean responses (delay) more predictable. Therefore, high values and/or frequent outliers would signal lack of synchrony between space/place and time dimensions in an intervention.

Table. Outliers per scheduler/device.
Device # exp Type scheduler # outliers # outliers positive # outliers negative
BQ Aquaris V #1 Advanced 1646 925 721
BQ Aquaris V #2 Basic 2996 2499 497
Honor 9 #2 Advanced 307 209 98
Honor 9 #1 Basic 5033 5018 15
Motorola Moto G #2 Advanced 2861 2852 9
Motorola Moto G #1 Basic 4377 4377 NA
Nvidia Shield Tablet #1 Advanced 294 285 9
Nvidia Shield Tablet #2 Basic 3 3 NA
Xiaomi Mi A1 #1 Advanced 224 121 103
Xiaomi Mi A1 #2 Basic 28 27 1

Observations:

  • Advanced scheduler beats the basic one in terms of number of outliers. Honor 9 shows a remarkly difference.

  • Majority of outliers are positive (beyond top whisker).

BQ Aquaris V

  • Number of outliers: 4642

  • Number of outliers (top range): 3424 - 73.8% (3.424/4.642).

  • Number of outliers (bottom range): 1218 - 26.2% (1.218/4.642).

Nvidia Shield Tablet

  • Number of outliers: 297

  • Number of outliers (top range): 288 - 97% (288/297).

  • Number of outliers (bottom range): 9 - 3% (9/297).

Xiaomi Mi A1

  • Number of outliers: 252

  • Number of outliers (top range): 148 - 58.7% (148/252).

  • Number of outliers (bottom range): 104 - 41.3% (104/252).

Honor 9

  • Number of outliers: 5340

  • Number of outliers (top range): 5227 - 97.9% (5.227/5.340).

  • Number of outliers (bottom range): 113 - 2.1% (113/5.340).

Motorola Moto G

  • Number of outliers: 7238

  • Number of outliers (top range): 7229 - 99.9% (7.229/7.238).

  • Number of outliers (bottom range): 9 - 0.1% (9/7.238).

Which one of the analysed schedulers is more accurate? (Distribution of delay)

Basic assumption: Higher accuracy = lower delay

To evaluate it, we plot timestamps as follows (being “n” each row in the data file):

  • planning timestamp(n) - execution timestamp(n) [the current one, which displays framework overhead]
  • planning timestamp(n-1) - planning timestamp(n) [in order to detect wrong planning timestamp values]
  • execution timestamp(n-1) - execution timestamp(n) [this last one should be reliable]

Analsysis of consecutive planning timestamps

We calculate planning timestamp(n-1) - planning timestamp(n) to detect wrong planning timestamp values.

## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
Table. Descriptive statistics of ‘planning timestamp(n-1) - planning timestamp(n)’ [in order to detect wrong planning timestamp values] per scheduler/device.
# exp Type scheduler Device # records Start End Min Bottom whisker 1st Qu. Median Mean 3rd Qu. Top whisker Max
#1 Advanced BQ Aquaris V 21391 2020-03-30 12:15:03 2020-04-14 13:07:57 0 60.159 60.463 60.565 60.735 60.666 60.970 1323.965
Nvidia Shield Tablet 21453 2020-03-30 12:15:03 2020-04-14 13:08:14 0 58.629 60.012 60.179 60.560 60.980 62.415 191.099
Xiaomi Mi A1 21465 2020-03-30 12:23:27 2020-04-14 13:07:50 0 58.936 60.039 60.224 60.501 60.796 61.887 628.508
Basic Honor 9 21574 2020-03-30 12:00:03 2020-04-14 13:01:35 0 59.907 60.014 60.025 60.243 60.103 60.235 296.462
Motorola Moto G 21619 2020-03-30 12:00:07 2020-04-14 13:01:16 0 59.980 60.008 60.016 60.117 60.027 60.055 120.719
#2 Advanced Honor 9 19377 2020-04-15 11:00:02 2020-04-29 12:59:15 0 59.426 60.063 60.109 62.794 60.738 61.748 44935.248
Motorola Moto G 19735 2020-04-16 12:09:53 2020-04-30 12:58:44 0 59.733 60.032 60.064 61.441 60.377 60.895 186.625
Basic BQ Aquaris V 20146 2020-04-15 11:16:26 2020-04-29 13:00:20 0 60.045 60.247 60.262 60.351 60.385 60.592 94.369
Nvidia Shield Tablet 3528 2020-04-15 11:16:18 2020-04-17 22:27:06 0 59.113 60.002 60.099 60.388 60.979 62.029 64.777
Xiaomi Mi A1 10832 2020-04-15 11:16:23 2020-04-23 00:52:00 0 59.285 60.009 60.269 60.352 60.579 61.429 168.009
Table. Outliers of consecutive planning timestamps per scheduler/device.
Device Type scheduler # outliers # outliers positive # outliers negative
BQ Aquaris V Advanced 1698 888 810
BQ Aquaris V Basic 3085 2517 568
Honor 9 Advanced 455 263 192
Honor 9 Basic 5066 5048 18
Motorola Moto G Advanced 2589 2560 29
Motorola Moto G Basic 3653 3287 366
Nvidia Shield Tablet Advanced 197 181 16
Nvidia Shield Tablet Basic 4 3 1
Xiaomi Mi A1 Advanced 359 180 179
Xiaomi Mi A1 Basic 21 19 2

Execution timestamps

  • execution timestamp(n-1) - execution timestamp(n) [this last one should be reliable]
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
## Adding missing grouping variables: `scheduler`
Table. Descriptive statistics of ‘execution timestamp(n-1) - execution timestamp(n)’ per scheduler/device.
# exp Type scheduler Device # records Start End Min Bottom whisker 1st Qu. Median Mean 3rd Qu. Top whisker Max
#1 Advanced BQ Aquaris V 21391 2020-03-30 12:15:03 2020-04-14 13:07:57 45.580 60.157 60.462 60.565 60.739 60.666 60.971 1324.016
Nvidia Shield Tablet 21453 2020-03-30 12:15:03 2020-04-14 13:08:14 -103.246 58.723 60.002 60.182 60.565 60.982 62.392 190.955
Xiaomi Mi A1 21465 2020-03-30 12:23:27 2020-04-14 13:07:50 -64.584 59.123 60.042 60.232 60.504 60.804 61.888 628.652
Basic Honor 9 21574 2020-03-30 12:00:03 2020-04-14 13:01:35 0.566 59.843 60.010 60.028 60.246 60.122 60.290 355.912
Motorola Moto G 21619 2020-03-30 12:00:07 2020-04-14 13:01:16 0.089 59.744 59.976 60.023 60.120 60.131 60.363 120.747
#2 Advanced Honor 9 19377 2020-04-15 11:00:02 2020-04-29 12:59:15 -70.030 59.090 60.062 60.118 62.797 60.744 61.758 44935.732
Motorola Moto G 19735 2020-04-16 12:09:53 2020-04-30 12:58:44 -101.278 59.101 60.024 60.080 61.444 60.648 61.573 225.801
Basic BQ Aquaris V 20146 2020-04-15 11:16:26 2020-04-29 13:00:20 59.542 60.043 60.247 60.262 60.354 60.383 60.587 94.369
Nvidia Shield Tablet 3528 2020-04-15 11:16:18 2020-04-17 22:27:06 59.113 59.113 59.997 60.099 60.405 60.979 62.062 64.704
Xiaomi Mi A1 10832 2020-04-15 11:16:23 2020-04-23 00:52:00 26.323 59.305 60.021 60.269 60.358 60.578 61.402 168.004

Framework overhead

  • execution timestamp(n-1) - execution timestamp(n) [this last one should be reliable]

BQ Aquaris V

How does delay behave in the middle 50% of data (IRQ)?

Advance scheduler shows worse delay times. In addition, the central cluster shows steadily increased values of delay over time. In contrast, the basic scheduler shows uniform delays over time adn concentrated aoru 0.3 sec. Teh basic scheduler’s boxplot shows a thinner box than the advance scheduler’s. There seems no clear relationship between battery levels and delays, which suggests that low battery levels do not impact (or at least not significally) on the planned execution time of a task.

What happens when battery is going to be fully recharged?

Honor 9

How does delay behave in the middle 50% of data (IRQ)?

## Warning: Removed 31 row(s) containing missing values (geom_path).

## Warning: Removed 22 row(s) containing missing values (geom_path).

Motorola Moto G

See the shiny app! (TO BE UPDATED)

Does the battery level influence timely alarm execution? (Distribution of delay over battery - without outliers)

Which one of the analysed schedulers loses fewer alarm executions?

Higher reliability = higher percentage of alarms executed / expected alarms for the experiment time frame

Boxplots

Distribution of data based on a five number summary (“minimum”, first quartile (Q1), median, third quartile (Q3), and “maximum”).

Basic statistics applied to delay and device_name. Given a device, how do the box plots change on each scheduler?